The present invention primarily relates to replay detection techniques.
The amount of video content is expanding at an ever increasing rate, some of which includes sporting events. Simultaneously, the available time for viewers to consume or otherwise view all of the desirable video content is decreasing. With the increased amount of video content coupled with the decreasing time available to view the video content, it becomes increasingly problematic for viewers to view all of the potentially desirable content in its entirety. Accordingly, viewers are increasingly selective regarding the video content that they select to view. To accommodate viewer demands, techniques have been developed to provide a summarization of the video representative in some manner of the entire video. Video summarization likewise facilitates additional features including browsing, filtering, indexing, retrieval, etc. The typical purpose for creating a video summarization is to obtain a compact representation of the original video for subsequent viewing.
There are two major approaches to video summarization. The first approach for video summarization is key frame detection. Key frame detection includes mechanisms that process low level characteristics of the video, such as its color distribution, to determine those particular isolated frames that are most representative of particular portions of the video. For example, a key frame summarization of a video may contain only a few isolated key frames which potentially highlight the most important events in the video. Thus some limited information about the video can be inferred from the selection of key frames. Key frame techniques are especially suitable for indexing video content but are not especially suitable for summarizing sporting content.
The second approach for video summarization is directed at detecting events that are important for the particular video content. Such techniques normally include a definition and model of anticipated events of particular importance for a particular type of content. The video summarization may consist of many video segments, each of which is a continuous portion in the original video, allowing some detailed information from the video to be viewed by the user in a time effective manner. Such techniques are especially suitable for the efficient consumption of the content of a video by browsing only its summary. Such approaches facilitate what is sometimes referred to as “semantic summaries”.
Babaguchi et al. propose a technique to link live and replay scenes in American football broadcast video. The replay scenes are detected by the sandwiching of digital video effects (DVEs) on either side of the replay segment. Babaguchi et al. note that it is impossible to design a general detector applicable to all DVEs, because there are a considerable number of DVE patterns that appear in everyday broadcasts. The DVE effect taught by Babaguchi et al. is a gradual shot change operation related to spatial aspects. The linking of live and replay scenes is performed based upon the dominant color of the key frame and the ratio of the number of vertical lines to that of horizontal lines on the field, which is representative of the camera angle. In effect, Babaguchi et al. detect replay segments by detecting special transition effects, by the manipulation of two natural scenes from the game, usually as a unique type of wipe. The technique for linking using the technique taught by Babaguchi et al., namely using the ratio, is suitable for the intended application, namely, American Football. In addition, the use of motion vectors and segmented spatial regions are inherently unreliable, and the technique can not automatically detect the transitions if the features of the transitions are not previously known.
Pan et al. propose a technique to detect slow-motion replay segments in sports video for highlights generation. The technique localizes semantically important events in sport programs by detecting slow motion replays of these events, and then generates highlights of these events at multiple levels. A hidden Markov model is used to model slow motion replays, and an inference algorithm computes the probability of a slow motion replay segment, and localizes the boundaries of the segment as well. The technique characterizes the pattern of slow motion as discontinuity on pixel wise intensity-based differences of adjacent frames that could be described by three features, and characterizes boundaries of replays segments as another feature: video-evolution ratio that is extracted from color histograms. The hidden-Markov model is applied to make the final decision on slow motion and the boundary of segments containing slow motion replay. However, the technique taught by Pan et al. is limited only to the detection of slow motion replay segments. The system taught by Pan et al. is unable to detect replay segments that are played back at the regular video rate, which are common in sports broadcast.
What is desired, therefore, is a robust replay detection technique.